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Abstract 

Many Embedded Systems ai"e indeed Software Based Control Systems, 
that is control systems whose controller consists of control software run- 
ning on a microcontroller device. This motivates investigation on Formal 
Model Based Design approaches for automatic synthesis of embedded sys- 
tems control software. This paper addresses control softwai^e synthesis for 
discrete time nonlinear systems. We present a methodology to overapprox- 
imate the dynamics of a discrete time nonlinear hybrid system T-L by means 
of a discrete time linear hybrid system in such a way that controllers 
for JZ-^ are guaranteed to be controllers for Ti. We present experimental re- 
sults on the inverted pendulum, a challenging and meaningful benchmark in 
nonlinear Hybrid Systems control. 
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1 Introduction 



Many Embedded Systems are indeed Software Based Control Systems (SBCSs). 
An SBCS consists of two main subsystems: the controller and the plant. Typ- 
ically, the plant is a physical system consisting, for example, of mechanical or 
electrical devices whereas the controller consists of control software running on a 
microcontroller. In an endless loop, the controller reads sensor outputs from the 
plant and sends commands to plant actuators in order to guarantee that the closed 
loop system (that is, the system consisting of both plant and controller) meets 
given safety and liveness specifications {System Level Formal Specifications). 

Software generation from models and formal specifications forms the core of 
Model Based Design of embedded software ll2n . This approach is particularly 
interesting for SBCSs since in such a case system level (formal) specifications are 
much easier to define than the control software behavior itself. 

The typical control loop skeleton for an SBCS is the following. Measure x of 
the system state from plant sensors go through an analog-to-digital (AD) conver- 
sion, yielding a quantized value x. A function Ctrl Region checks if x belongs 
to the region in which the control software works correctly. If this is not the 
case a Fault Isolation and Recovery (FDIR) procedure is triggered, otherwise a 
function ctrlLaw computes a command u to be sent to plant actuators after a 
digital-to-analog (DA) conversion. Basically, the control software design prob- 
lem for SBCSs consists in designing software implementing functions ctrlLaw 
and ctrlRegion. 

For SBCSs, system level specifications are typically given with respect to 
the desired behavior of the closed loop system. The control software (that is, 
ctrlLaw and ctrlRegion) is designed using a separation-of-concerns approach. 
That is. Control Engineering techniques (e.g., see [[TOl ) are used to design, from 
the closed loop system level specifications, /wncfzona/ specifications {control law) 
for the control software whereas Software Engineering techniques are used to de- 
sign control software implementing the given functional specifications. Such a 
separation-of-concerns approach has several drawbacks. 

First, usually control engineering techniques do not yield a formally verified 
specification for the control law when quantization is taken into account. This is 
particularly the case when the plant has to be modelled as a Hybrid System, that is 
a system with continuous as well as discrete state changes JSlIBIIllSl. As a result, 
even if the control software meets its functional specifications there is no formal 
guarantee that system level specifications are met since quantization effects are 
not formally accounted for. 



2 



Second, issues concerning computational resources, such as control software 
Worst Case Execution Time (WCET), can only be considered very late in the 
SBCS design activity, namely once the software has been designed. As a result, 
the control software may have a WCET greater than the sampling time. This inval- 
idates the schedulability analysis (typically carried out before the control software 
is completed) and may trigger redesign of the software or even of its functional 
specifications (in order to simplify its design). 

Last, but not least, the classical separation-of-concems approach does not ef- 
fectively support design space exploration for the control software. In fact, al- 
though in general there will be many functional specifications for the control soft- 
ware that will allow meeting the given system level specifications, the software 
engineer only gets one to play with. This overconstrains a priori the design space 
for the control software implementation preventing, for example, effective per- 
formance trading (e.g., between number of bits in AD conversion, WCET, RAM 
usage, CPU power consumption, etc.). We note that the above considerations also 
apply to the typical situation where Control Engineering techniques are used to 
design a control law and then tools like Simulink are used to generate the control 
software. 

The previous considerations motivate research on Software Engineering meth- 
ods and tools focusing on control software synthesis (rather than on control law 
synthesis as in Control Engineering). The objective is that from the plant model 
(as a hybrid system), from formal specifications for the closed loop system be- 
havior and from Implementation Specifications (that is, number of bits used in the 
quantization process) such methods and tools can generate correct-by-construction 
control software satisfying the given specifications. 

The tool QKS Il23]| synthesise control software for Discrete Time Linear Hy- 
brid Systems (DTLHSs). However, the dynamics of many interesting hybrid sys- 
tems cannot be directly modeled by linear predicates. The focus of the present 
paper is control software synthesis for nonlinear Discrete Time Hybrid Systems. 

1.1 Our Main Contributions 

We model the controlled system (plant) as a Discrete Time Hybrid System (DTHS), 
that is a discrete time hybrid system whose dynamics is modeled as a predicate 
(possibly non linear) over a set of continuous as well as discrete variables that 
describe system state, system inputs and disturbances. 

System level safety as well as liveness specifications are modeled as sets of 
states defined, in turn, as predicates. In our setting, as always in control problems, 
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liveness constraints define the set of states that any evolution of the closed loop 
system should eventually reach (goal states). Using an approach similar to the 
one in ||20| . in ||24| it has been proven that both existence of a controller and 
existence of a quantized controller for DTHSs are undecidable problems, even 
for very restricted classes of DTHSs. Accordingly, we can only hope for non 
complete or semi-algorithms. 

In this paper we present a general approach to deal with discrete time non- 
linear hybrid systems. The basic idea is to overapproximate the behaviour of a 
DTHS H by means of a DTLHS Stemming from Corollary [3l that ensures 
that controllers for Cji are guaranteed to be controllers for T-L, we synthesize con- 
trol software by giving as input to the tool QKS [(231 the linear plant model 
the desired quantization schema, and system level formal specifications. 

Since dynamics overapproximates the dynamics of H, the controllers that 
we synthesize are inherently robust, that is they meet the given closed loop re- 
quirements notwithstanding nondeterministic small disturbances such as varia- 
tions in the plant parameters. Tighter overapproximations makes finding a con- 
troller easier, whereas coarser overapproximations makes controllers more robust. 
As in the linear case, the automatically generated software has a Worst Case Ex- 
ecution Time (WCET) guaranteed to be linear in the number of bits of the state 
quantization schema. Moreover, control software computes commands in such a 
way that the closed loop system follows a (near) time optimal strategy to reach the 

goal [ini. 

We present experimental results on the inverted pendulum benchmark [|22]| . a 
challenging and well studied example in control synthesis. 

1.2 Related Work 

Control Engineering has been studying control law design (e.g., optimal control, 
robust control, etc.), for more than half a century (e.g., see ifTOl ). Also Quan- 
tized Feedback Control has been widely studied in control engineering (e.g. see 
lfT4]| ). However such research does not address hybrid systems (our case) and, 
as explained above, focuses on control law design rather than on control soft- 
ware synthesis (our goal). Furthermore, all control engineering approaches model 
quantization errors as statistical noise. As a result, correctness of the control law 
holds in a probabilistic sense. Here instead, we model quantization errors as non- 
deterministic {malicious) disturbances. This guarantees system level correctness 
of the generated control software (not just that of the control law) with respect to 
any possible sequence of quantization errors. 
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When the plant model is a Linear Hybrid Automaton (LHA) [[BUI reachability 
and existence of a control law are both undecidable problems [[T9ll20| . This, of 
course, has not prevented devising effective (semi) algorithms for such problems. 
Examples are in flU [161 [IS [301 12S1 0. Control software synthesis for continu- 
ous time linear systems (no switching) has been implemented in the tool Pessoa 
[[25l . Such an approach exploits suitable finite state abstraction (e.g. see |[27ll26l ) 
to synthesize a control law computing commands from real valued state measures 
(no quantization). The control software is then generated by passing to Simulink 
such a control law. In the same wavelength, OTI generates a control strategy from 
a finite abstraction of a Piecewise Affine Discrete Time Hybrid Systems (PWA- 
DTHS). Also the Hybrid Toolbox [Q considers PWA-DTHS. Such a tool outputs 
a feedback control law that is then passed to Matlab in order to generate control 
software. Finite horizon control of PWA-DTHS has been studied using a MILP 
based approach. See, for example, [8J. Explicit finite horizon control synthesis al- 
gorithms for discrete time (possibly non-linear) hybrid systems have been studied 
in [[T2]| and citations thereof. 

We note that all such approaches do not account for state feedback quanti- 
zation since they all assume exact (i.e. real valued) state measures. Thus, as 
explained above, they do not offer any formal guarantee about system level cor- 
rectness of the generated software, which is instead our focus here. 

Quantization can be seen as a sort of abstraction, which has been widely stud- 
ied in a hybrid system formal verification context (e.g., see |[2l[3]|). Note however 
that in a verification context abstractions are designed so as to ease the verifica- 
tion task whereas in control software synthesis quantization is a design require- 
ment since it models a hardware component (AD converter) which is part of the 
specification of the control software synthesis problem. Indeed, in our setting, we 
have to design a controller notwithstanding the nondeterminism stemming from 
the quantization process. As a result, the techniques used to devise clever abstrac- 
tions in a verification setting cannot be directly used in our synthesis setting where 
quantization is given. 

The tool QKS |[23ll synthesize control software from system level specification 
for Discrete Time Linear Hybrid Systems whenever a a constructive sufficient 
condition for control software existence holds. Here, we address control software 
synthesis for a more general class of discrete time hybrid systems. 

In the context of Hybrid Systems verification, the overapproximation of Hy- 
brid Systems with Linear Hybrid Systems has been studied in [[TSlI and [[TTll . Such 
works consider dense time models, and focus on verification rather than control 
synthesis. Moreover, we observe that we can obtain tighter approximations, since 
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DTLHSs allow us to model system dynamics with predicates that mix present and 
next state variables. 

Correct-by-construction software synthesis in a finite state setting has been 
studied, for example, in |l6l |29l [HI. Such approaches cannot be directly used in 
our context since they cannot handle continuous state variables. 

Summing up, to the best of our knowledge, no previously published result is 
available about automatic generation of correct-by-construction control software 
from a DTHS model of the plant, system level formal specifications and imple- 
mentation specifications {quantization, that is number of bits in AD conversion). 

2 Background 

We denote with [n] an initial segment { 1 , . . . , n} of the natural numbers. We de- 
note withX = \xi,...,Xn^di finite sequence (list) of variables. By abuse of language 
we may regard sequences as sets and we use u to denote list concatenation. Each 
variable x ranges on a known (bounded or unbounded) interval either of the 
reals or of the integers (discrete variables). We denote with Vx the set Ylxex^x- 
To clarify that a variable x is continuous (i.e. real valued) we may write x''. Sim- 
ilarly, to clarify that a variable x is discrete (i.e. integer valued) we may write 
x^. Analogously X'' (X'^) denotes the sequence of real (integer) variables in X. 
Finally, boolean variables are discrete variables ranging on the set B = {0, 1 }. Ifx 
is a boolean variable we write x for (1 - .^:). 

2.1 Predicates 

An expression E{X) over a list of variables X is an expression of the form 
Xi/£[„] <3i/;(X), where fi{X) is a possibly nonlinear function over X and at are 
rational constants. E{X) is a linear expression if each fi{X) is a projection (i.e. 
fi{X) =Xi), i.e. if it is a linear combination of variables Y,i£[n\ <^i^i- A constraint is 
an expression of the form E{X) <b, where is a rational constant. In the follow- 
ing, we also write E{X)>b iox -E{X) < -b. 

Predicates are inductively defined as follows. A constraint C{X) over a list of 
variables X is a predicate over X. If A{X) and B{X) are predicates over X, then 
(A(X) /\B{X)) and (A(X) w B{X)) are predicates over X. Parentheses may be 
omitted, assuming usual associativity and precedence rules of logical operators. A 
conjunctive predicate is a conjunction of constraints. For conjunctive predicates 
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we will also write: E{X)=b for {{E{X) <b) a {E{X)>b)) and a < x < for 
xia A x<b, where x^X. 

A valuation over a list of variables X is a function v that maps each variable 
X e X to a value v{x) e V^. Given a valuation v, we denote with X* e Vx the 
sequence of values [v{x\) , . . . ,v{xn)'\. By abuse of language, we call valuation 
also the sequence of values X* . A satisfying assignment to a predicate P over Z 
is a valuation X* such that P{X*) holds. If a satisfying assignment to a predicate 
P over X exists, we say that P h feasible. Abusing notation, we may denote with 
P the set of satisfying assignments to the predicate P{X). Two predicates P and 
Q over X are equivalent, denoted by P = Q, if they have the same set of satisfying 
assignments. Two predicates P and Q are equisatisfiable if P is feasible iff Q is 
feasible. 

A variable x € X is said to be bounded in P if there exist a, b eV^ such that 
P(Z) implies a<x<b. A predicate P is bounded if all its variables are bounded. 

Given a constraint C{X) and a fresh boolean variable (guard) y^X, the guarded 
constraint y C{X) (if j then C{X)) denotes the predicate {{y = 0) vC(X)). 
Similarly, we use y C{X) (if not y then C(X)) to denote the predicate {{y - 
1) vC(X)). A guarded predicate is a conjunction of either constraints or guarded 
constraints. It is possible to show that, if a guarded predicate P is bounded, then 
P can be transformed into an equivalent (bounded) conjunctive predicate fi24il . 

2.2 Labeled Transition Systems 

A Labeled Transition System (LTS) is a tuple S = (5,A,r) where 5 is a (possibly 
infinite) set of states, A is a (possibly infinite) set of actions, and T : 5 x A x 
5 -> B is the transition relation of S. We say that T (and S) is deterministic if 
T{s,a,s') AT{s,a,s") implies s' = s" , and nondeterministic otherwise. Let 5 e 5 
and a e A. We denote with Adm(iS,5') the set of actions admissible in s, that is 
Adm((5,5') = {(3 e A I 35' : T{s,a,s')} and with Img(iS,5,a) the set of next states 
from s via a, that is lmg{S,s,a) = {s' \ T{s,a,s')}. A run or path for an LTS S 
is a sequence 71 = so,ao,si,ai,S2,a2: ... of states 5? and actions at such that V? > 
T(st,at,St+i). The length |7r| of a finite run tt is the number of actions in n. We 
denote with n^^^ (t) the (t + 1 )-th state element of n, and with 7r(^) (t) the (f + 1 )-th 
action element of n. That is n^^\t) = St, and 7r(^)(f) = a/. 

Given two LTSs Si = (5, A, Ti) and 1S2 = (S, A, 72), we say that Si refines 
S2 (notation Si E ^2) iff Ti{s,a,s') implies T2(s,a,s') for each state s,s' eS and 
action aeA. The refinement relation is a partial order on LTSs. 
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2.3 LTS Control Problem 



A controller for an LTS S is used to restrict the dynamics of S so that all states in 
the initial region will reach in one or more steps the goal region. In the following, 
we formalize such a concept by defining strong solutions to an LTS control prob- 
lem. In what follows, let S = (5,A,r) be an LTS, I, G ^ S be, respectively, the 
initial and goal regions of S. 

Definition 1 A controller /or S is a function ^ : S x A ^ B such that V5 e S, Va e 
A, if K{s,a) then 35' T{s,a,s'). dom(^) denotes the set of states for which at 
least a control action is enabled. Formally, dom(^) = {5 € 5 | 3a K{s^a)}. iS^^) 
denotes closed loop system, that is the LTS {S,A,T^^^), where T^^\s,a,s') = 
r (5, a, 5') A ^(5, a). 

We call a path n fullpath [[6l if either it is infinite or its last state 7t('^)(|7r|) has 
no successors (i.e. Adm(5,7r('^)(|7r|)) = 0). We denote with Path(5',a) the set 
of fuUpaths starting in state s with action a, i.e. the set of fuUpaths 71 such that 
7i(^)(0) =5and 7r(^)(0) =a. 

Given a path n in S, we define 7(5, 71, G) as follows. If there exists n > s.t. 
7r(^) (n) e G, then 7(5, 7r, G) = mm{n | n > a ni^) g Qy Otherwise, 7(5, 7r, G) = 
+00. We require n>0 since our systems are nonterminating and each controllable 
state (including a goal state) must have a path of positive length to a goal state. 
Taking sup0 = +00 and inf0 = -00, the worst case distance of a state s from 
the goal region G is 7strong(5,G,5) = sup{7^(5, G,5,a) | a e Adm(5,5)}, being 
Js{S,G,s,a) = sup{7(5,G,7r) | n e Path(5,a)}. 

Definition 2 A control problem /or S is a triple V = (5,/,G). A strong solution 
(or simply a solution) to V is a controller Kfor S, such that I £ dom(^) and for 
all s e Dom(^), Jstrong{S^^^ , G, s) is finite. 

An optimal solution to V is a solution K* to V s.t. for all solutions K to V, 
for all s eVx we have: Jstrong{S^^ \ G,s) < Jstrong{'S^'^\G,s). The most general 
optimal (mgo) solution to V is an optimal solution K to V s.t. for all optimal 
solutions K to V, for all s e Vx, for all u e Vu we have: K(s,u) K(s,u). It is 
easy to see that this definition is well posed (i.e., the mgo solution is unique) and 
that K does not depend on I. 
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3 Discrete Time Hybrid Systems 



In this section we introduce our class of Discrete Time Hybrid Systems (DTHS 
for short), together with the DTHS representing the inverted pendulum on which 
our experiments will focus. Moreover, we will define in Sect. 13 .21 the Quantized 
Control Problem. 

Definition 3 A Discrete Time Hybrid System is a tuple T-L = {X,U,Y, N) where: 

• X = X''uX^ is a finite sequence of real (X^) and discrete (X'^) present state 
variables. We denote with X' the sequence of next state variables obtained 
by decorating with ' all variables in X. 

• U = Vuif^ is a finite sequence of input variables. 

• y = yuy^ is a finite sequence of auxiliary variables. Auxiliary variables 
are typically used to model modes (e.g., from switching elements such as 
diodes) or "local" variables. 

• N{X ,U ,Y,X') is a conjunctive predicate over X uU uY uX' defining the 
transition relation (next state) of the system. N is deterministic ifN{x,u,yi ,x') 
A N{x, u,y2,x") implies x' = x", and nondeterministic otherwise. 

A DTHS is bounded if the predicate N is bounded. A DTHS is deterministic if 
N is deterministic. A DTHS is linear, and we call it DTLHS ifN is a conjunction 
of linear constraints. 

Since any bounded guarded predicate can be transformed into a conjunctive 
predicate (see Sect. 12.11) . for the sake of readability we will use bounded guarded 
predicates to describe the transition relation of bounded DTHSs. To this aim, we 
will also clarify which variables are boolean, and thus may be used as guards in 
guarded constraints. 

Example 1 Let us consider a simple inverted pendulum l[22^ . as shown in Fig. \J} 
The system is modeled by taking the angle and the angular velocity as state 
variables. The input of the system is the torquing force u, that can infiuence the 
velocity in both directions. Moreover, the behaviour of the system depends on the 
pendulum mass m, the length of the pendulum I and the gravitational acceleration 
g. Given such parameters, the motion of the system is described by the differential 
equation = j sin0 + 
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Figure 1: Inverted Pendulum with Stationary Pivot Point. 



In order to obtain a state space representation, we consider the following nor- 



The DTHS model % for the pendulum is the tuple {X ,U ,Y,N), where X = 
{xi,X2} is the set of continuous state variables, U = {u} is the set of input vari- 
ables, and 7=0. Differently from l[22\l . we consider the problem of finding a dis- 
crete controller, whose decisions maybe "apply the force clockwise" (u= \), "ap- 
ply the force counterclockwise " (u = -!)", or "do nothing " (u = 0). The intensity of 
the force will be given as a constant F . Finally, the discrete time transition relation 
N is obtained from the equations in ([7]) by introducing a constant T that models the 
sampling time. N is the predicate (xj -xi + Tx2) a {x'j = X2 + T j sinxi + T-^Fu). 

The semantics of DTHSs is given in terms of LTSs. 

Definition 4 Let % = (X, U, Y, N) be a DTHS. The dynamics ofH is defined by 
the Labeled Transition System LTS('H) = (Vx, T>u, N) where: N : T>x x Vy x 
Vx B is a function s.t. N{x,u,x') = 3y e Vy : N{x,u,y,x'). A state xfor % is a 
state xfor LTS(?{) and a run (or pathj/or % is a run for LTS(?{) (Sect. \2.2\) . 



A DTHS control problem (H,/, G) is defined as theLTS control problem (LTS('H), 
/, G). To accommodate quantization errors, always present in software based 
controllers, it is useful to relax the notion of control solution by tolerating an 



malized system, where x\ is the angle and X2 is the angular speed 0. 




(1) 



3.1 DTHS Control Problem 
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(arbitrarily small) error e on the continuous variables. This leads to the defi- 
nition of £-solution. Let e be a nonnegative real number, c x Z'". The 
Z-relaxation of W is the set {ball of radius £) Bz{W) = {(zi,...z„, qi,...q,n) \ 
3{xx,...,x„,qi,...q,n) e WandVze[n] \zi-Xi\<£.}. 

Definitions Let ('HJ,G) be a DTHS control problem and Z be a nonnegative 
real number. An £ solution to {'H,I,G) is a solution to the LTS control problem 
(LTS(?^),/,Se(G)). 

Example 2 Let T be a positive constant ( sampling time ). We define the DTHS T-L 
- ({x}, {m}, 0, A^) where x is a continuous variable, u is a boolean variable, and 
N{x,u,x') = [u^x' =x+{^-x)T]a[u^x' =x+{x-^)T]. LetV = (U, L G) be 
a control problem, where I = -2 <x <2.5, and G = x = 0. A controller may drive 
the system near enough to the goal x-0, by enabling a suitable action in such a 
way that x' <x when x > and x' > x when x <0. If the sampling time T is small 
enough with respect to £ (for example T < the controller: K{x,u) = (-2 < 
x<0 A u) V (0 < X < ^ am) V < X < 2.5 a u) is an e solution to G). 
Observe that, that any controller K' such that ^'(|,0) holds is not a solution, 
because since N{j,0,^) holds, the closed loop system 'H(^) may loop forever 
along the path | , 0, | , . . .. 

Example 3 The typical goal for the inverted pendulum in ExampleUlis to turn the 
pendulum steady to the upright position, starting from any possible initial posi- 
tion, within a given speed interval. In our experiments, the goal region is defined 
by the predicate G{X) = (-p <xi < p) a (-p <X2< p), where p € {0.05,0.1}, and 
the initial region is defined by the predicate I(X) = (-71 <xi <%) a (-4 <X2< 4j. 

3.2 Quantized Control Problem 

In order to manage real variables, in classical control theory the concept of quanti- 
zation is introduced (e.g., see lfT4ll ). Quantization is the process of approximating 
a continuous interval by a set of integer values. In the following we formally 
define a quantized feedback control problem for DTHSs. 

A quantization function J for a real interval I =[a,b]is a non-decreasing func- 
tion y. I ^ 7j s.t. y(/) is a bounded integer interval. We will denote y(/) as 
/ = [y{o) ^y{b)'\. The quantization step of y, notation ||y||, is defined as sup{ \w- 
z\\w,z € / Ay(w) = y(z)}. For ease of notation, we extend quantizations to integer 
intervals, by stipulating that in such a case the quantization function is the identity 
function. 
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Definition 6 Let U = {X ,U ,Y ,N) be a DTHS, andletW = XyjU uY . A quantiza- 
tion Qfor % is a pair (A,r), where: 

• A is a predicate over W that explicitely bounds each variable in W. For each 
w eW, we denote with A^ its admissible region and with Aw - Ilweiy^vv 

• T is a set of maps T - {^^ | w e V7 and 7^ is a quantization function for A^^?^. 

Let V7 = [wi, . . .Wyt] and v - [vi, . . .v^] e Ajy. Wfe write T{y)for the tuple [y^j (vi), 
. . . , Yvv^ {yk)\ Finally, the quantization step ||r|| is defined as sup{ ||y|| | ye F}. 

A control problem admits a quantized solution if control decisions can be made 
by just looking at quantized values. This enables a software implementation for a 
controller. 

Definition? Let % = {X,U,Y,N) be a DTHS, Q = (A,r) be a quantization for 
% and V - {T-L,I,G) be a DTHS control problem. A Q Quantized Feedback 
Control (QFC) solution to V is a ||F|| solution K{x,u) to V such that K{x,u) = 
K(r(x),r{u)) where K:r{Ax)><r{Au) ^M. 

Example 4 Let V be as in Example^ Let us consider the quantization (A,F) 
where A = I and F = {jx} where Jxi^) = The set T(Ax) of quantized states 
is the integer interval [-2,2]. No Q QFC solution can exist, because defining 
both ^(1,1) and ^(1,0) allows infinite loops to be potentially executed in the 
closed loop system. Of course, the controller K in Example^can be obtained as 
a quantized controller decreasing the quantization step, for example by taking F 
= {%} where %{x) = [Sx\. 

4 DTLHS overapproximation of DTHSs 

In (231, we presented the tool QKS that given a DTLHS control problem V - 
(T-LjfG) and a quantization schema as input, yields as output control software 
implementing a most general optimal quantized controller for V, whenever a suf- 
ficient condition holds. In this section we show how a DTHS H can be overap- 
proximate by a DTLHS C-^, in such a way that LTS('H) E LTS(£^). The follow- 
ing theorem ensures that controllers for £^ are guaranteed to be controllers for 

n. 
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4.1 DTHS linearization 



Let C(y), with V ^XuU u Y uX', be a constraint in A'^ that contains a nonlinear 
function as a subterm. Then C{V) has the shape f{R,W) +E{V) < b, where R^V 
is a set of n real variables {ri , . . . , r„}, and V7 £ is a set of discrete variables. 
For each w e Vw, we define the function fw{R) obtained from /, by instanciat- 
ing discrete variables with w, i.e fw{R) = f{R,w). Then C{V) is equivalent to 
the conjunctive predicate AweVwifwiR) +E(y) < b]. In order to make the over- 
approximation tighter, we partition the domain of each function fw{R) into 
m hyperintervals /i , /2 . . . /,„ , where Ii = Ilj^^,^^[a^j,b'j]. In the following Relj will 
denote the conjunctive predicate Aje[ii] ^) - ^ ^'j- 

Let f^,i{R) and f^^,^{R) be over- and under- linear approximations of fw{R) 
over the hyperinterval i.e. such that R e /, implies /^^,(^) < fw{R) ^ fwi(^)- 
Taking |Pvk| ^ ^ fresh continuous variables Y = {yw,i}we'DwMn]^ define the con- 
junctive predicate C(y, 7 ) : 

[y.,i+Eiv)<b] 

By introducing \Vyy \ x n fresh boolean variables Z = {z/}vi;£D^.;e[,7] , C{V,Y) can 
be translated into the following equisatisfiable conjunctive predicate C{V,Y,Z): 

bw,/+£(v)<z>] 

A Aw€©^ A/cH zw,/ ^ fw,i{R) < yw,i < f^^iiR) 

A AweDiv A;e[m] ^w,! ^ ^ /,• A Aw€'Dw T,i£[m] Zw,i ^ 1 

As a result, this transformation eliminates a nonlinear subexpression of a con- 
straint C(\/) and yields a constraint C(\/,y,Z) such that 3Y,Z[C{V,Y,Z) ^C{V)]. 
Given a DTHS ^ = {X,U,Y,N), without loss of generality, we may suppose that 
the transition relation A^^ is a conjunction Aie[m]Ci{X ,U ,Y,X') of constraints. By 
applying the above transformation to each nonlinear subexpressions occurring in 
A^, we obtain a conjunction of linear constraints A'^ = Aie[m]Ci{X ,U ,Y ,X'), such 
that iV ^ A^. Hence, starting from a DTHS we find a DTLHS Ch = {X,UJ,N), 
whose dynamics overapproximate the dynamics of H. 

Theorem 1 Let Ti = {X ,U ,Y,N) be a DTHS and let Cq-i be its linearization. Then 
we have that LTS{T-L) ^LTS^Cy). 
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Figure 2: Linearization of sinx in [-71, 7i]. 

Theorem 2 Let S\ = {S,A,T\) and S2 = (5, A, two LTSs, and let K be a 
solution for the LTS control problem (S2,I,G). If Si refines S2 and for all s 
Adm(Si,s) =Adm(Si,s), then K is a solution also for (i5i,/,G). 

Proof 1 (Sketch) The proof is by induction on n = /strong (152 ,G,s). Ifn=\ and 
K{s,a), then lmg{S2,s,a) £ G. Since Si E S2, we also have that lmg(Si,s,a) £ 
lmg{S2,s,a) £ G. Moreover, Adm{Si,s) =Adm(S2,s) implies that there exists at 
least a transition of the shape Ti(s,a,s') with s' eG and thus Jsmng{<S\ ,G,s) = 1 

too. This implies that {s \ JsimngiS ^^\g,s) = 1} = {s\ Jsirong{<S2^\ G,s) = 1}. The 
inductive step is similar, by substituting G with the set of states {s \ Jsmng{S2, G,s) - 
n-l}. 

Corollary 3 Let % = {X,U,Y,N) be a DTHS and let £^ be its linearization. Let 
K be a solution for the DTLHS control problem G). Then K is a solution 

also for the DTHS control problem (J-L,I,G). 

Example 5 The DTHS % = {X ,U ,0,N) model for the inverted pendulum in Ex. Ul 
contains the nonlinear function sinxi. We define the linearization = (X, i7, Y,N) 
as follows. In order to exploit sinus periodicity, we consider the equation x\ = 
2%yk + ya> where y^ represents the period in which xi lies andya e [-7^,7^] repre- 
sents the actual x\ inside a given period. 
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Figure 3: Controllable re- Figure 4: Trajectories for Figure 5: Same trajecto- 
gion for F - 0.5, T - 0.1, 7^(^05^) and TiS^oT) start- ries of Fig. |4]in the phases 
andZj = 9. ing from (;ci,;c2) = (71,0). space. 

This allows us to apply our linearization to e [-71,71] only. We partition the 
interval [-71, 7l] into four sub-intervals I\, h, I3, h cls shown in Fig. |2] For y^. e 
I\ = [-71, -|] we define fi(ya) (^s the line passing through points (-7r,sin(-7r)) 
and (-|,sin(-|)), i.e. f^{ya) = -0.6369ya + 2. Moreover, we define /f (ja) '^s 
the line which is tangent to the curve sinja I\ medium point, i.e. /f (ja) - 
0.7073(jcx -I- 0.785) -0.7068. Functions and are obtained analogously. 

Finally, we have that Y = y^uy'" = {yk,yq,zi,Z2,Z3,Z4} {ya} and N = {x[ = 
xi+2iiyq + Tx2) A {x'2=X2 + Tfya + T-^Fu)Axi = 2nyk+ya/\ A%iZi ^ f[ <ya< 
ft^AUz,^xieI,AZ%,z,>l. 

4.2 Linearization: a systematic approach 

When nonlinear subexpressions are functions, a systematic approach to com- 
pute linear overapproximations of a DTHS makes use of Taylor polinomial of 
degree 1 as piecewise affine functions that over- and under- approximate the value 
of a function. Let f{x) be a function of n real variables over a given interval 
/. By Taylor's theorem, we may derive linear under- and over-approximations for 
f{x) around a given point e / as follows. Namely, we have that exists t e [0, 1] 
such that f{x) = /{xq) + \/f{xo){x-xo) + j{x-xoYH{x+t{x-xo)){x-xo), being 
H the Hessian matrix of /. If we know two real numbers m and M that are the 
minimum and the maximum value of j{x-xo)^H{x + t{x-xo)){x-xo),ma given 
interval around xq. In this case we can choose f^{x) = f{xo) + s7f{xo){x-xo) +M 
and f-{x) = f{xo) + \7f{xo){x-xo)+m. 
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5 Experimental Results 



In this section we present our experiments that aim at evaluating effectiveness of 
our linearization technique. 

5.1 Experimental Settings 

We present experimental results obtained by using QKS [|23ll on the inverted pen- 
dulum described in Example [B In order to let QKS handle such a case study, we 
linearize the DTHS % in Example [U with the DTLHS of Example |5l In all 
our experiments, as in Il22l we set parameters I and m in such a way that j = 1 (i.e. 
l = g) and ^ = ^ (i-e. m = As for the quantization, we set A^, = [-I.Itt, l.lTt] 
and = [-4,4], and we define A = A^i X ^ All. Moreover, we use uniform 
quantization functions dividing the domain of each state variable (xi,X2) into 2^ 
equal intervals, where b is the number of bits used by AD conversion. The re- 
sulting quantization is Qt, = (A,r^), with ||r/,|| = ^. Since we have two quantized 
variables (xi,X2) each one with b bits, the number of quantized (abstract) states is 
exactly 2-^'\ Finally, the initial region / and goal region G are as in Ex.[3l thus the 
DTHS [DTLHS] control problem we consider is P = {H, I, G) [(£^, /, G)]. 

We run QKS for different values of the remaining parameters, i.e. F (force 
intensity), p (goal tolerance), T (sampling time), and b (number of bits of AD). 
For each of such experiments, QKS outputs a control software ^ in C language. 
In the following, we sometimes make explicit the dependence on F and b by 
writing K^^ . In order to evaluate performance of K, we use an inverted pendulum 
simulator written in C. The simulator computes the next state by using Eq. ([T]) 
of Ex. [H thus simulating a path of 'H(^). Such simulator also implements the 
following features: 

• random disturbances (up to 4%) in the next state computation are intro- 
duced, in order to assess K robustness w.r.t. non-modelled disturbances; 

• Eq. ([T]) is translated into the discrete time version by means of a simulation 
time step much smaller than the sampling time T used in % (and C%). 
Namely, = 10"^ seconds, whilst T - 0.01 or T = 0.1 seconds. This allows 
us to have a more accurated simulation. Accordingly, K is called each lO'* 
(or 10^) simulation steps of %. When K is not called, the last chosen action 
is selected again {sampling and holding). 
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All experiments have been carried out on an Intel(R) Xeon(R) CPU @ 2.27GHz, 
with 23GiB of RAM, Kernel: Linux 2.6.32-5-686-bigmem, distribution Debian 
GNU/Linux 6.0.3 (squeeze). 



5.2 Underactuated Inverted Pendulum (F = 0.5) 

In order to stabilize an underactuated inverted pendulum (i.e. when F < 1) from 
the hanging position to the uprigth position, a controller needs to find a non ob- 
vius strategy that consists of swinging the pendulum once or more times to gain 
enough momentum. We show that QKS is able to synthesize such a controller by 
running it on where F = 0.5 (note that in ll22l F = 0.7). Results are in Tab. [H 
where each row corresponds to a QKS run. Columns meaning in Tab. [T] are as 
follows. Columns b, T and p show the corresponding inverted pendulum parame- 
ters. Column 1^1 shows the size of the C code for k'^}^ . Finally, columns CPU and 
RAM show the computation time (in seconds) and RAM usage (in KB) needed 

by QKS to synthesize 

As for Kq'^^ performance, it is easy to show that by reducing the sampling time 

(b) 

T and the quantization step (i.e. increasing b), we increase the quality of Kq g in 
terms of ripple, set-up time and coverage. In fact. Fig. |4] shows the simulations 
ofn^^o.s) and (^0.5 ). 

As we can see, Kq drives the system to the goal with 
a smarter trajectory, with one swing only. This have a significant impact on the 
set-up time (the system stabilizes after about 8 seconds when controlled by K^^^ 
instead of about 10 seconds required when controlled by K^^^). Fig. [3] shows 

that the controllable region of K^^^ (i.e., dom(^Q^g )) covers almost all states in 
the admissible region that we consider. Different colors mean different set of 
actions enabled by the controller. We observe that the mgo solution enables more 
than one action in a significant portion of the controllable region. The control 
software, however, is generated in such a way that one action is chosen in each 

state. Finally, Fig. [10] shows the ripple o 
such ripple is very low (0.018 radiants). 



state. Finally, Fig.[TO]shows the ripple of xi for H^^o-s inside the goal. Note that 



5.3 Very Underactuated Inverted Pendulum (F = 0.3) 

We succeeded to find controllers for the inverted pendulum for any value of F 
down to 0.3, with T = 0.1 seconds and p = 0.1. However, simulations show that 
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Figure 6: Simulation for Figure 7: States turned di- ^. „ „ 

, , , . , ^ Figure 8: States turned di- 
Ti^'^i ) starting from rectly to the goal with f = „ , ,u i -.u r o 

rectly to the goal with F -2. 

the behaviour of the resulting closed loop system is somewhat puzzling. As it is 

shown in Fig. [6] for ^*^^o.3 ), after three swings the pendulum is correctly driven 
to the goal, but at that point the controller is not able to maintain the plant inside 
the goal. In fact, the controller let the pendulum fall and makes it do a complete 
round in order to reach again the upright position. This behaviour is repeated 27 

times, before the ^03^^ makes pendulum stabilize into the goal region. 

As already noted in ||22]| . all controllers for underactuated pendulum use two 
very different strategies to stabilize the system depending on the initial state. 
When the angle is positive and the speed is negative (and in a suitable range that 
depends on F), the controller turns directly the pendulum into the upright posi- 
tion. Symmetrically, this also happens when the angle is negative and the speed is 
positive. Otherwise the controller let the pendulum fall down to gain enough mo- 
mentum (or to smoothly slow down it). Therefore, starting from very near states 
may lead the system to follow very different trajectories. Reducing F squeezes 
the region of states from which the pendulum is directly turned into the upright 
position. As Fig. |7] shows, when F is equal to 0.3, we have a rather pathological 
situation: the frontier between the two strategies lies inside the goal region. The 
controller sometimes is unable to keep the system inside the goal, because distur- 
bances introduced by the simulator make the system cross the frontier between 
the two strategies. When this frontier lies far enough from the goal (see Fig.[8]for 
the case F = 2), this phenomenon is essentially harmless and leads, at worst, to 
suboptimal strategies. 

5.4 Overactuated Pendulum {F = 2) 

When F is greater than 1, finding a control strategy is less challenging. It is worth 
noting however that, even in this case, our approach allows us to find controllers 
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Table 1: Experimental Results for inverted pendulum with f = 0.5. 



b 


T 
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\K\ 


CPU 


MEM 
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0.1 


0.1 


2.73e+04 


2.56e+03 


7.72e+04 


9 


0.1 


0.1 


5.94e+04 


1.13e+04 


l.lOe+05 


10 


0.1 


0.1 


1.27e+05 


5.39e+04 


1.97e+05 


11 


0.01 


0.05 


4.12e+05 


1.47e+05 


2.94e+05 



that hardly can be synthesized by means of traditional analytical methods. In 
Fig. |9l we show trajectories in the phases space of H^^2 ) with T = 0.01 sec- 
onds, p = 0.05, and starting values for x\ are in {f , f , ^,3} and X2 = 0. H^'^'^ ^ 

follows highly non-smooth trajectories: ^2^^"* drives the system along an optimal 
approach to the goal. Before joining this ideal path to the goal, the controller, 
in order to optimize the set up time, drives the system at the maximum possible 
"cruising" speed that allows the pendulum to be stopped in the goal. For higher 
values of F, this cruising speed is even higher. 

6 Conclusions 

We presented an automatic methodology to sinthesize control software for nonlin- 
ear Discrete Time Hybrid Systems. The control software is correct-by-construction 
with respect both System Level Formal Specifications of the closed loop system 
and Implementation Specification, namely the quantization schema. Our exper- 
imental results on the inverted pendulum benchmark show the effectiveness of 
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our approach and that we synthesize near optimal controllers that hardly can be 
designed by using traditional analytical methods of Control Engineering. 

The present work can be extended in several directions. First of all, it would be 
interesting to consider control synthesis of controllers that are optimal with respect 
a cost function given as input of the control problem, rather than simply time- 
optimal. Another natural possible future research direction is to investigate fully 
symbolic control software synthesis algorithms based, for example, on efficient 
quantifier elimination procedures, in order to efficiently deal with Hybrid Systems 
with several continuous state variables. 
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